Apparatus for transmitting image data

ABSTRACT

An image data transmitting apparatus is disclosed, the apparatus being configured to receive a right image and a left image and transmit the received right image and left image to a host system using a USB method, such that a depth image and a color image can be compressed in real time by a standard protocol, the compressed depth image and the color image are respectively converted to USB packet data, and the converted depth image and the color image are transmitted to a host system.

TECHNICAL FIELD

The teachings in accordance with exemplary embodiments of this inventionrelate generally to an apparatus for transmitting image data(hereinafter referred to as “image data transmission apparatus”, orsimply “apparatus”).

BACKGROUND ART

Generally, a camera system outputting a depth image using a stereocamera transmits two image data of depth image and color image to a hostsystem. FIG. 1 is a configurative block diagram illustrating an imagedata transmitting apparatus according to prior art.

Referring to FIG. 1, a conventional image data transmitting apparatus(100) is configured such that a right image of M×N and a left image ofM×N are image-processed by an image processor (110) to output a depthimage of M×N and a color image of M×N, where frame size andsynchronization of the depth image and color image are corrected by aninterface (120), which is simultaneously outputted to a host system(200) such as a television or a computer (PC).

In the conventional image data transmitting apparatus (100), when theinterface unit (120) transmits an image to the host system (200), theimage is transmitted to the host system (200) directly, or via IEED1394,CAMLINK, HDMI (High-Definition Multimedia Interface) or USB (UniversalSerial Bus). The CAMLINK or HDMI is limited in use due to high price,such that an USB type interface is generally used except for handhelddevices. Currently, USB 2.0 type is used.

However, the USB-type interface unit (120) suffers from a disadvantagein that frame size is restricted due to bandwidth limitation (480 Mbps)in transmitting a depth image and a color image. Furthermore, theUSB-type interface unit (120) suffers from a disadvantage in that thehost system (200) tends to be complicated, because one endpoint is usedfor transmitting an image data (see FIG. 1), a non-standard encoder (notshown) is required for transmitting two image data through onetransmission path, and the host system (200) requires a non-standarddecoder (not shown) for decoding the coded image data.

Meanwhile, the host system (200) also suffers from disadvantages in thatit must be mounted with separate non-standard driver software forrecognizing devices and a separate non-standard API (ApplicationProgramming Interface) to go with the separate non-standard driversoftware.

DISCLOSURE OF INVENTION Technical Problem

Accordingly, an exemplary embodiments of the present invention mayrelate to an image data transmission apparatus that substantiallyobviates one or more of the above disadvantages/problems due tolimitations and disadvantages of related art, and it is an object of anexemplary embodiment of the present invention to provide an apparatusfor transmitting image data (image data transmission apparatus, orsimply apparatus) configured to perform, by an interface unit, astandard image compression encoding, to increase a data transmissionefficiency, and to cope with bandwidth limitation of USB type.

It is another object of an exemplary embodiment of the present inventionto provide an image data transmission apparatus configured to use twoendpoints of USB interface unit to transmit two images (depth image andcolor image) to a host system via each endpoint, whereby separateencoding and decoding are not required.

It is another object of an exemplary embodiment of the present inventionto provide an image data transmission apparatus configured to support aUSB video class to dispense with a separate device driver, where a hostsystem processes an image in response to an API relative to the USBvideo class to dispense with an additional API.

Technical problems to be solved by the present invention are notrestricted to the above-mentioned description, and any other technicalproblems not mentioned so far will be clearly appreciated from thefollowing description by skilled in the art.

Solution to Problem

In order to accomplish the above objects, in one general aspect of anexemplary embodiment the present disclosure, there may be provided animage data transmission apparatus, the apparatus comprising: acompression unit performing compression on the depth image data and thecolor image data in real time using a predetermined standard protocol; aconversion unit converting the compressed depth image data and the colorimage data to USB packet data respectively; and a transmission unittransmitting the converted depth image data and the color image data toa host system via USB.

In an exemplary embodiment of the present disclosure, the standardprotocol used by the compression unit may include JPEG or H.242.

In an exemplary embodiment of the present disclosure, the host systemmay decompress the received depth image data and the color image data,using the standard protocol used by the compression unit.

In an exemplary embodiment of the present disclosure, the transmissionunit may transmit the converted depth image data and the color imagedata by allocating to mutually different endpoints.

In an exemplary embodiment of the present disclosure, the host systemmay selectively or simultaneously use the received depth image data andthe color image data.

In an exemplary embodiment of the present disclosure, the transmissionunit may transmit the depth image data and the color image data using auniversal transport protocol.

In an exemplary embodiment of the present disclosure, the universaltransport protocol may include a UVC (Universal Video Class).

In an exemplary embodiment of the present disclosure, the host systemmay further comprise a UVC driver.

In an exemplary embodiment of the present disclosure, the apparatus mayfurther comprise an image processor outputting the depth image data andthe color image data from a right image and left image in a stereo type.

In an exemplary embodiment of the present disclosure, the imageprocessor may comprise a stereo matching unit determining an imagedistance by receiving the right image and the left image, and a linebuffer outputting the color image data from the right image and the leftimage, and outputting the depth image data from an output of the stereomatching unit.

In another general aspect of an exemplary embodiment of the presentinvention, there may be provided an image data transmission apparatusconfigured to receive a right image and a left image and transmit thereceived right image and left image to a host system using a USB type,the apparatus comprising: an image processor determining a depth imagedata and a color image data using a right image and a left image; and aUSB interface unit compressing the depth image data and the color imagedata in real time using a standard protocol, and transmitting thecompressed depth image data and the color image data respectively to ahost system via USB.

In an exemplary embodiment of the present disclosure, the USB interfaceunit may comprise a compression unit performing compression on the depthimage data and the color image data in real time using a standardprotocol; a conversion unit converting the compressed depth image dataand the color image data to USB packet data respectively; and atransmission unit transmitting the converted depth image data and thecolor image data.

In an exemplary embodiment of the present disclosure, the transmissionunit may transmit the converted depth image data and the color imagedata by allocating to mutually different endpoints.

In an exemplary embodiment of the present disclosure, the transmissionunit may transmit the depth image data and the color image data using auniversal transport protocol.

In an exemplary embodiment of the present disclosure, the universaltransport protocol may include a UVC.

Advantageous Effects of Invention

One advantageous effect is that a standard type such as JPEG or H.264 isused to perform a standard image compression to increase transmissionefficiency and to overcome a bandwidth limitation of USB type.

Another advantageous effect is that two endpoints of USB interface unitare used to transmit two images (depth image and color image) to a hostsystem via each endpoint, whereby separate encoding and decoding are notrequired to simplify the system.

Still another advantageous effect is that a standard UVC is used totransmit image data, whereby the host system needs no separate devicedriver, and the host system processes the image data in response to APIrelative to the UVC, whereby additional API is not required to simplifythe system.

Still further advantageous effect is that a camera can be easily usedfree from installation of additional software.

BRIEF DESCRIPTION OF DRAWINGS

The teachings of the present invention can be readily understood byconsidering the following detailed description in conjunction with theaccompanying drawings, in which:

FIG. 1 is a configurative block diagram illustrating an image datatransmitting apparatus according to prior art.

FIG. 2 is a configurative block diagram illustrating an image datatransmitting apparatus according to an exemplary embodiment of thepresent invention;

FIG. 3 is a configurative block diagram illustrating an image processorof FIG. 2 according to an exemplary embodiment of the present invention;

FIG. 4 is a detailed configurative block diagram illustrating a USBinterface unit of FIG. 2 according to an exemplary embodiment of thepresent invention; and

FIG. 5 is a detailed configurative block diagram illustrating a hostsystem of FIG. 2 according to an exemplary embodiment of the presentinvention.

BEST MODE FOR CARRYING OUT THE INVENTION

The following description is not intended to limit the invention to theform disclosed herein. Consequently, variations and modificationscommensurate with the following teachings, and skill and knowledge ofthe relevant art are within the scope of the present invention. Theembodiments described herein are further intended to explain modes knownof practicing the invention and to enable others skilled in the art toutilize the invention in such, or other embodiments and with variousmodifications required by the particular application(s) or use(s) of thepresent invention.

The disclosed embodiments and advantages thereof are best understood byreferring to FIGS. 1-5 of the drawings, like numerals being used forlike and corresponding parts of the various drawings. Other features andadvantages of the disclosed embodiments will be or will become apparentto one of ordinary skill in the art upon examination of the followingfigures and detailed description. It is intended that all suchadditional features and advantages be included within the scope of thedisclosed embodiments, and protected by the accompanying drawings.Further, the illustrated figures are only exemplary and not intended toassert or imply any limitation with regard to the environment,architecture, or process in which different embodiments may beimplemented. Accordingly, the described aspect is intended to embraceall such alterations, modifications, and variations that fall within thescope and novel idea of the present invention.

It will be understood that the terms “includes” and/or “including” whenused in this specification, specify the presence of stated features,regions, integers, steps, operations, elements, and/or components, butdo not preclude the presence or addition of one or more other features,regions, integers, steps, operations, elements, components, and/orgroups thereof. That is, the terms “including”, “includes”, “having”,“has”, “with”, or variants thereof are used in the detailed descriptionand/or the claims to denote non-exhaustive inclusion in a manner similarto the term “comprising”.

Furthermore, “exemplary” is merely meant to mean an example, rather thanthe best. It is also to be appreciated that features, layers and/orelements depicted herein are illustrated with particular dimensionsand/or orientations relative to one another for purposes of simplicityand ease of understanding, and that the actual dimensions and/ororientations may differ substantially from that illustrated. That is, inthe drawings, the size and relative sizes of layers, regions and/orother elements may be exaggerated or reduced for clarity. Like numbersrefer to like elements throughout and explanations that duplicate oneanother will be omitted. Now, the present invention will be described indetail with reference to the accompanying drawings.

The terminology used herein is for the purpose of describing particularembodiments only and is not intended to be limiting of the generalinventive concept. As used herein, the singular forms “a”, “an” and“the” are intended to include the plural forms as well, unless thecontext clearly indicates otherwise.

Now, the image data transmitting apparatus according to exemplaryembodiments of the present invention will be described in detail withreference to the accompanying drawings.

FIG. 2 is a configurative block diagram illustrating an image datatransmitting apparatus according to an exemplary embodiment of thepresent invention.

Referring to FIG. 2, an image data transmitting apparatus (1) accordingto an exemplary embodiment of the present invention comprises an imageprocessor (10) and a USB (Universal Serial Bus) interface unit (20),where an output of the USB interface unit (20) is transmitted to a hostsystem (30).

The image processor (10) having received stereo type image of a rightimage of pixels of M×N and a left image of pixels of M×N outputs a depthimage of pixels of M′×N′ and a color image of pixels of M×N. The USBinterface unit (20) performs an image compression encoding of theinputted depth image and color image in a standard protocol, allocatesmutually different endpoints (A, B) to the two images, and transmits theimages to the host system (30) via the endpoints. The transmission ofthe image data is compliant with the UVC (USB Video Class).

The host system (30) is equipped with a UVC driver, and receives twoimage data from the image data transmission apparatus (1) according toan exemplary embodiment of the present invention, which allows two videodevices to be recognized. A user can selectively use one image or twoimages as necessary, and no additional APIs are required by the hostsystem (30). Hereinafter, the image data transmission device (1) will bedescribed in more detail with reference to the accompanying drawings.

FIG. 3 is a configurative block diagram illustrating an image processorof FIG. 2 according to an exemplary embodiment of the present invention.

Referring to FIG. 3, the image processor (10) in the image datatransmitting apparatus (1) according to an exemplary embodiment of thepresent invention comprises a stereo matching unit (11), delaying units(12, 13) and a line buffer (14).

The stereo matching unit (11) receives a right image of M×N (41) and aleft image of M×N (42) from cameras (not shown) to determine a distanceto an image captured by the cameras. Algorithms used for determining thedistance by the stereo matching unit (11) includes dynamic programmingand SAD (Sum of Absolute Difference). The algorithms used by the stereomatching unit (11) according to an exemplary embodiment of the presentinvention are not limited thereto, and the distance to the image can bedetermined by various types of algorithms.

A depth image (43) determined by the stereo matching unit (11) isoutputted by passing through the line buffer (14), and the right image(41) and the left image (42) are respectively delayed for apredetermined time (Δt) by the delaying units (12, 13) to be outputtedin color images via the line buffer (14).

The depth image (43) outputted by the image processor (10) according toan exemplary embodiment of the present invention may be M′×N′ in size,and configured in a plurality of frames. For example, the depth image(43) may be an image data whose bit number per pixel is 8 bits.Furthermore, the color image (44) outputted by the image processor (10)according to an exemplary embodiment of the present invention may be M×Nin size, and configured in a plurality of frames. For example, the colorimage (44) may be an image data whose bit number per pixel is 16 bits.

The image size may be changed in response to a processing speed of theimage processor (10) according to an exemplary embodiment of the presentinvention. For example, the image size appropriate for the system may beminimally 640×480 pixels, and as the image size increases, quality ofproduct can be enhanced. In case of size of the image being 640×480pixels, a minimum data amount can be expressed by the following table:

TABLE 1 Bit per Frame Resolution (pixels) pixel rate bandwidth Depthimage 307,200(640 × 480)  8 bits 30fps  9.2 Mbyte/sec Color image307,200(640 × 480) 16 bits 30fps 18.4 Mbyte/sec

That is, a data amount actually transmitted for transmitting an imagesize of 640×480 pixels is minimally 27.6 Mbyte/sec, such that datacompression is required for transmitting an image data to the hostsystem (30) through the USB interface unit (20) according to anexemplary embodiment of the present invention.

Mode for the Invention

FIG. 4 is a detailed configurative block diagram illustrating a USBinterface unit of FIG. 2 according to an exemplary embodiment of thepresent invention, where the USB interface unit (20) according to anexemplary embodiment of the present invention may be configured with oneIC (Integrated Circuit).

Referring to FIG. 4, the USB interface unit (20) according to anexemplary embodiment of the present invention comprises an imagereceiving unit (21), a standard image compression unit (22), a packetconversion unit (23) and a transmitting unit (24).

The image receiving unit (21) receives a depth image and a color imagefrom the image processor (10). The standard image compression unit (22)performs real-time image data compression, and performs an individualcompression on the respective images (depth image and color image) usingthe standard protocols of JPEG (Joint Photographic Experts Group) orH.264. The JPEG and H.264 are video compression standards. It should beapparent to the skilled in the art that the JPEG and H.264 are videocompression standards that are well known, such that no more detailedexplanation thereto will be omitted. However, the video compressionstandards that are used by the standard image compression unit (22)according to an exemplary embodiment of the present invention are notlimited to the JPEG and H.264, and therefore, it should be apparent tothe skilled in the art that use of newly established standard protocolsare not ruled out.

The packet conversion unit (23) converts the compressed image data toUSB packet data individually to each image data for transmitting theimage data compressed by the standard image compressing unit (22).

The transmitting unit (24) transmits the USB packet data to the hostsystem (30). The transmitting unit (24) transmits the USB packet data oftwo images by allocating to mutually different USB endpoints (A, B),whereby the host system (30) can effectively separate two images (depthimage and color image).

Meanwhile, although the conventional interface unit as shown in FIG. 1has transmitted image data to the host system (30) using non-standarddata protocol, the transmitting unit (24) according to an exemplaryembodiment of the present invention can use the universal image datatransmission protocols. For example, the universal image datatransmission protocols defined by the Window system or Linux system canbe utilized. A representative example is the UVC. As noted from theabove description, the USB interface unit (20) according to an exemplaryembodiment of the present invention uses the universal protocols toallow dispensing with a separate USB device to be used by the hostsystem (30).

FIG. 5 is a detailed configurative block diagram illustrating a hostsystem of FIG. 2 according to an exemplary embodiment of the presentinvention, where only constituent parts related to processing of imagedata transmitted by the image data transmitting apparatus (1) accordingto an exemplary embodiment of the present invention are described, suchthat understanding of the host system (30) configured only with theconstituent parts illustrated in FIG. 5 should be avoided.

Referring to FIG. 5, the host system (30) receiving image data from theimage data transmitting apparatus (1) according to an exemplaryembodiment of the present invention comprises a UCV driver unit (31), astandard image decompression unit (32) and an application unit (33),which are mounted as defaults in a conventional host systems (e.g.,notebook computers, TVs, tablet computers, mobile terminals).

The host system (30) receives two types of image data (depth image andcolor image) from the USB interface unit (20) via a receiver (notshown), whereby it is possible to recognize images received from the UVCdriver unit (31) as being the images received from two video devices.This can be enabled by use of two endpoints by the USB interface unit(20) according to an exemplary embodiment of the present invention.

The standard image decompression unit (32) decompresses the image datacompressed by the standard image compression unit (22) of the USBinterface unit (20). It should be apparent that the decompression usingthe compression method used by the standard image compression unit (22)of the USB interface unit (20) is well known to the skilled in the art.

The application unit (33) may be, for example, an API, and providesapplication for use by a user using the image data, whereby the user canselectively use one or two images as necessary.

As apparent from the foregoing, the image data transmitting apparatusaccording to an exemplary embodiment of the present invention isadvantageous in that data transmission efficiency can be enhanced by thestandard image compression unit using the standard protocol such as JPEGor H.264, and limitation of bandwidth using the USB type can beovercome.

Furthermore, the image data transmitting apparatus according to anexemplary embodiment of the present invention is advantageous in thattwo endpoints of the USB interface unit (20) are used to transmit twoimages (depth image and color image) to the host system through eachendpoint, whereby there is no need of requiring separate encoding anddecoding processes.

Furthermore, the image data transmitting apparatus according to anexemplary embodiment of the present invention is advantageous in that animage data is transmitted using the standard UVC to dispense with aseparate driver unit in the host system, and an image signal isprocessed in response to API relative to the UVC, whereby additional APIis not required to simplify the system. Thus, the host system can easilyuse a camera from installation of additional software.

Meanwhile, the exemplary embodiments of the present disclosure may beembodied in the form of program code embodied in tangible media, such asmagnetic recording media, optical recording media, solid state memory,floppy diskettes, CD-ROMs, hard drives, or any other non-transitorymachine-readable storage medium. When the exemplary embodiments of thepresent disclosure are implemented using software, constituent means ofthe present disclosure may be code segments executing necessaryprocesses. The programs or code segments may be also embodied in theform of program code, for example, whether stored in a non-transitorymachine-readable storage medium, loaded into and/or executed by amachine, or transmitted over some transmission medium or carrier, suchas over electrical wiring or cabling, through fiber optics, or viaelectromagnetic radiation, wherein, when the program code is loaded intoand executed by a machine, such as a computer, the machine becomes anapparatus for practicing the disclosure.

The above-described embodiments of the present invention can also beembodied as computer readable codes/instructions/programs on a computerreadable recording medium. Examples of the computer readable recordingmedium include storage media, such as magnetic storage media (forexample, ROMs, floppy disks, hard disks, magnetic tapes, etc.), opticalreading media (for example, CD-ROMs, DVDs, etc.), carrier waves (forexample, transmission through the Internet) and the like. The computerreadable recording medium can also be distributed over network coupledcomputer systems so that the computer readable code is stored andexecuted in a distributed fashion.

The previous description of the present invention is provided to enableany person skilled in the art to make or use the invention. Variousmodifications to the invention will be readily apparent to those skilledin the art, and the generic principles defined herein may be applied toother variations without departing from the spirit or scope of theinvention. Thus, the invention is not intended to limit the examplesdescribed herein, but is to be accorded the widest scope consistent withthe principles and novel features disclosed herein.

INDUSTRIAL APPLICABILITY

The image data transmitting apparatus according to exemplary embodimentof the present invention has an industrial applicability in that astandard protocol such as JPEG or H.264 is used to perform a standardimage compression to increase transmission efficiency and to overcome abandwidth limitation of USB method, two endpoints of USB interface unitare used to transmit two images (depth image and color image) to a hostsystem via each endpoint, whereby separate encoding and decoding are notrequired to simplify the system, a standard UVC is used to transmitimage data, whereby the host system needs no separate device driver, andthe host system processes the image data in response to API relative tothe UVC, whereby additional API is not required to simplify the system,and a camera can be easily used free from installation of additionalsoftware.

1. An image data transmission apparatus, the apparatus comprising: acompression unit performing compression on the depth image data and thecolor image data in real time using a predetermined standard protocol; aconversion unit converting the compressed depth image data and the colorimage data to USB packet data respectively; and a transmission unittransmitting the converted depth image data and the color image data toa host system via USB.
 2. The apparatus of claim 1, wherein the standardprotocol used by the compression unit is JPEG or H.242.
 3. The apparatusof claim 1, wherein the host system decompresses received depth imagedata and the color image data, using the standard protocol used by thecompression unit.
 4. The apparatus of claim 1, wherein the transmissionunit transmits the converted depth image data and the color image databy allocating to mutually different endpoints.
 5. The apparatus of claim1, wherein the host system selectively or simultaneously uses thereceived depth image data and the color image data.
 6. The apparatus ofclaim 1, wherein the transmission unit transmits the depth image dataand the color image data using a universal transport protocol.
 7. Theapparatus of claim 6, wherein the universal transport protocol includesa UVC (Universal Video Class).
 8. The apparatus of claim 7, wherein thehost system further comprises a UVC driver.
 9. The apparatus of claim 1,further comprising an image processor outputting the depth image dataand the color image data from a right image and left image in a stereotype.
 10. The apparatus of claim 9, wherein the image processorcomprises a stereo matching unit determining an image distance byreceiving the right image and the left image, and a line bufferoutputting the color image data from the right image and the left image,and outputting the depth image data from an output of the stereomatching unit.
 11. An image data transmission apparatus, the apparatuscomprising: an image processor determining a depth image data and acolor image data using the right image and the left image; and a USBinterface unit compressing the depth image data and the color image datain real time using a standard protocol and transmitting the compresseddepth image data and the color image data respectively to a host systemvia USB.
 12. The apparatus of claim 11, wherein the USB interface unitcomprises a compression unit performing compression on the depth imagedata and the color image data in real time using a standard protocol; aconversion unit converting the compressed depth image data and the colorimage data to USB packet data respectively; and a transmission unittransmitting the converted depth image data and the color image data.13. The apparatus of claim 12, wherein the transmission unit transmitsthe converted depth image data and the color image data by allocating tomutually different endpoints.
 14. The apparatus of claim 12, wherein thetransmission unit transmits the depth image data and the color imagedata using a universal transport protocol.
 15. The apparatus of claim14, wherein the universal transport protocol includes a UVC.